{
"cells": [
{
"cell_type": "markdown",
"id": "586b41e3",
"metadata": {},
"source": [
"# 01 - Precipitation Reanalysis"
]
},
{
"cell_type": "markdown",
"id": "29a407b0",
"metadata": {},
"source": [
"## Learning goals of this module\n",
"- Learn how to do a basic analysis, comparing observed precipitation with reanalysis precipitation from RDPA and HRDPA\n",
"- Learn how to run a simple example to access and compare observed and reanalysis precipitation.\n",
"- Learn how to calculate and visualize simple error metrics\n",
"\n",
"## Assumptions\n",
"- We assume you are familiar with the concept of a __reanalysis__ product. \n",
"\n",
"### Reference to reanalysis products\n",
"\n",
"- [High Resolution Deterministic Precipitation Analysis (HRDPA)](https://open.canada.ca/data/en/dataset/eff69d42-ce81-4672-867f-cc3baaf4157a)\n",
"\n",
"- [Regional Deterministic Precipitation Analysis (RDPA)](https://www.canada.ca/en/environment-climate-change/services/climate-change/canadian-centre-climate-services/display-download/technical-documentation-regional-precipitation-analysis.html#toc0)\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "67c337fd",
"metadata": {},
"source": [
"## Run imports and set-up logging"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "fdf09f3b",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2026-06-18 22:29:50,163 - INFO - Running Veriflow version 0.1.0\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"The autoreload extension is already loaded. To reload it, use:\n",
" %reload_ext autoreload\n"
]
}
],
"source": [
"import logging\n",
"import sys\n",
"import warnings\n",
"from pathlib import Path\n",
"\n",
"from dotenv import load_dotenv\n",
"\n",
"from veriflow import run_pipeline\n",
"from veriflow.constants import VERSION\n",
"\n",
"# add project root (parent of notebook folder) to path\n",
"sys.path.append(str(Path(\"..\").resolve()))\n",
"\n",
"\n",
"from verification_plots import (\n",
" reanalysis_timeseries_plot,\n",
")\n",
"\n",
"# Reload automatically\n",
"%load_ext autoreload\n",
"%autoreload 2\n",
"\n",
"\n",
"warnings.filterwarnings(\"ignore\", category=RuntimeWarning)\n",
"warnings.filterwarnings(\"ignore\", category=FutureWarning)\n",
"\n",
"load_dotenv(dotenv_path=\"tutorial.env\", override=True)\n",
"\n",
"base_config = Path(\"config\")\n",
"base_config.exists()\n",
"\n",
"logging.basicConfig(\n",
" level=logging.INFO,\n",
" format=\"%(asctime)s - %(levelname)s - %(message)s\",\n",
" handlers=[logging.StreamHandler()],\n",
")\n",
"logging.info(f\"Running Veriflow version {VERSION}\")"
]
},
{
"cell_type": "markdown",
"id": "3017287e",
"metadata": {},
"source": [
"## Inspecting the _veriflow_ pipeline configuration\n",
"1. Open the config file in the \"config\" directory. The name of the file is identical to the name of the notebook.\n",
"2. Inspect each of the sections to gain an understanding of what this configuration is about.\n",
"\n",
"We are pulling from FEWS webservices (you will see this as downloads from urls)\n",
"- Station Observations\n",
"- RDPA Analysis at Stations\n",
"- HRDPA Analysis at Stations\n"
]
},
{
"cell_type": "markdown",
"id": "ec89e13b",
"metadata": {},
"source": [
"## Running the _veriflow_ pipeline"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "b339ac88",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2026-06-18 22:29:50,486 - INFO - Successfully initialized the configuration. \n",
"\t verification_period_start = 2026-05-15 00:00:00 \n",
"\t verification_period_end = 2026-06-08 00:00:00\n",
"2026-06-18 22:29:50,532 - INFO - Start getting data from FewsWebservice.\n",
"2026-06-18 22:29:51,712 - INFO - Successfully got data from FewsWebservice.\n",
"2026-06-18 22:29:51,714 - INFO - Start getting data from FewsWebservice.\n",
"2026-06-18 22:29:52,399 - INFO - Successfully got data from FewsWebservice.\n",
"2026-06-18 22:29:52,400 - INFO - Start getting data from FewsWebservice.\n",
"2026-06-18 22:29:52,946 - INFO - Successfully got data from FewsWebservice.\n",
"2026-06-18 22:29:52,950 - INFO - Successfully loaded all data from sources.\n",
"2026-06-18 22:29:52,983 - INFO - Successfully computed ContinuousScores for verification pair PC_RDPA.\n",
"2026-06-18 22:29:53,021 - INFO - Successfully computed ContinuousScores for verification pair PC_HRDPA.\n",
"2026-06-18 22:29:53,023 - INFO - Verification pipeline completed successfully.\n"
]
}
],
"source": [
"config_file = base_config / \"01_elbow_precipitation_analysis.yaml\"\n",
"ods = run_pipeline(config=(config_file, \"yaml\"))"
]
},
{
"cell_type": "markdown",
"id": "0d7caa3a",
"metadata": {},
"source": [
"## Evaluating the results in the _veriflow_ `OutputDataset`"
]
},
{
"cell_type": "markdown",
"id": "d178a47e",
"metadata": {},
"source": [
"\n",
"A good starting point for \"eyeball\" verification is simple: just looking at your observations and re-analysis in a visual way. Use the interactive elements in the plots below to zoom, pan and compare the results of our 2 re-analysis products."
]
},
{
"cell_type": "markdown",
"id": "840359c0",
"metadata": {},
"source": [
"### 1 - Check out the stations in the `OutputDataset`\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "ab797cc8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Stations: ['3031092' '3050778' 'MSC-005' '05BL813' '05BJ804' '05BL809' 'FIRES-B4'\n",
" '05BJ805' '05BL812' 'FIRES-B5' '05BJ806' '05BH803' '05BF825' '05BH802'\n",
" '05BL810' '05BF827']\n"
]
}
],
"source": [
"stations = ods.get(ods.verification_pairs[0]).coords[\"station\"].values\n",
"print(f\"Stations: {stations}\")"
]
},
{
"cell_type": "markdown",
"id": "5343e7a3",
"metadata": {},
"source": [
"### 2 - Visualize the precipitation (per station)\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "9b157dad",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
" \n",
" \n",
" "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"reanalysis_timeseries_plot(ods, station=stations[5])"
]
},
{
"cell_type": "markdown",
"id": "9ce2fd32",
"metadata": {},
"source": [
"### 3 - Visualize the precipitation (per station) and add the absolute error to the plot as well\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "6fb4781d",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"reanalysis_timeseries_plot(ods, station=stations[4], error_var=\"mae\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "veriflow",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}